The present invention relates to a drive method for a solid-state imaging device, a solid-state imaging device, and an imaging apparatus. More particularly, the invention relates to a drive method for an X-Y address solid-state imaging device, a typical example of which is a complementary metal-oxide semiconductor (CMOS) device image sensor, a solid-state imaging device implementing the above drive method, and an imaging apparatus using the solid-state imaging device.
The invention also pertains to a solid-state imaging apparatus and an imaging apparatus, and more particularly, to a solid-state imaging apparatus in which a color filter having a primary color component for generating luminance (Y) components and other color components is disposed on the surface of the pixels, and also to an imaging apparatus using the solid-state imaging apparatus as the imaging device.
To improve the frame rate in a solid-state imaging device, generally, the amount of pixel information is decreased by adding information concerning a plurality of pixels, as disclosed in, for example, Japanese Unexamined Patent Application Publication No. 2004-266369.
An example of the above-described technique is as follows. In color coding of a Bayer pattern shown in FIG. 1, from a 3×3 pixel area, the same color of pixels in the two columns and the two rows are extracted and added while shifting the 3×3 pixel area by three pixels by maintaining the original pixel pattern without changing the color spatial repeat pattern or changing the pixel pitch ratio in the vertical, horizontal, and oblique directions.
Red (R) pixels 311, 313, 331, and 333 located in the odd-numbered rows are added, and then, the resulting addition R signal is positioned at centroid A. Similarly, by horizontally shifting three pixels from the R pixels 311, 313, 331, and 333, green (G) pixels 314, 316, 334, and 336 are added, and then, the resulting addition G signal is positioned at centroid B. By further horizontally shifting three pixels from the G pixels 314, 316, 334, and 336, R signals 317, 319, 337, and 339 are added, and then, the resulting addition R signal is positioned at centroid C.
Then, by vertically shifting three pixels from the R pixels 311, 313, 331, and 333, G pixels 341, 343, 361, and 363 located in the even-numbered rows are added, and then, the resulting addition G signal is positioned at centroid D. By horizontally shifting three pixels from the G pixels 341, 343, 361, and 363, blue (B) pixels 344, 346, 364, and 366 are added, and then, the resulting addition B signal is positioned at centroid E. In this manner, by adding color pixels as described above over the entire pixel area, the same colors of pixels can be added while maintaining the original color pattern without changing the color spatial repeat pattern or changing the pixel pitch ratio in the vertical, horizontal, and oblique directions.
In imaging apparatuses, such as in digital still cameras and video cameras, the number of pixels of solid-state imaging apparatuses used as imaging devices is increasing, and solid-state imaging apparatuses having several millions of pixels are coming into widespread use. The use of multi-pixel imaging devices aims to obtain high-resolution images. However, there is still a demand for solid-state imaging apparatuses exhibiting higher resolution.
In single-panel digital cameras, the color pattern of a color filter used in a solid-state imaging apparatus is very important to obtain high resolution. A typical example of the color pattern is the known, widely used Bayer pattern.
Bayer Pattern
The Bayer pattern is a color pattern, as shown in FIG. 2, in which a GR line having G pixels and R pixels alternately and a GB line having G pixels and B pixels alternately are disposed alternately in the horizontal direction (also in the vertical direction). The feature of this Bayer pattern is that the pixels are disposed in a square lattice at regular intervals d (pixel pitches) of the pixels in the vertical and horizontal directions and that the ratio G:R:B of the GRB colors in this square lattice pattern is 2:1:1.
The spatial frequency characteristics of the RGB colors in the Bayer pattern are now described by separately considering the characteristics of the G color, which is the primary color for generating luminance (Y) components, and the other colors, i.e., the R and B colors.
Generally, the luminance signal Y is generated according to equation (1).Y=0.6G+0.3R+0.1B  (1)
Equation (1) is based on the fact that the human eye is more sensitive to the G color and less sensitive to the R and B colors. That is, if high resolution is necessary for the luminance (Y) components, it is very important to increase the resolution of the G color components, and not very high resolution is necessary for the other R and B color components.
FIGS. 3A and 3B illustrate the G pattern from which only G pixels are extracted from the Bayer pattern. The spatial frequency characteristics are now considered with reference to FIGS. 3A and 3B. If the pixel sampling rate is set to be the pixel pitch d, the sampling rate for the G pixels is equal to the pixel pitch d in the vertical and horizontal directions, and according to the sampling theorem, signal components having frequencies up to (½)fs (fs (=1/d): sampling frequency) can be collected. That is, it is possible to collect signal components indicated by the half-tone columns and the voided columns shown in FIG. 3A according to the theoretical threshold and it is not possible to collect signal components having higher frequencies beyond this threshold frequency.
Concerning the 45° oblique direction, since the sampling rate for the G pixels is d/√2, signal components up to (½√2)fs can be collected according to the sampling theorem.
Similarly, the spatial frequency characteristics of the R and B pixels are considered below. In this case, since the pixel pitches for the R and B pixels are the same, only the spatial frequency characteristics of the R pixels are described below.
The R pattern from which only the R pixels are extracted from the Bayer pattern is shown in FIGS. 3C and 3D. Concerning the spatial frequency characteristics of the R pixels, since the sampling rate for the R pixels is 2d in the vertical and horizontal directions, signal components having frequencies up to ¼fs can be collected according to the sampling theorem. In the oblique direction, the sampling rate for the R pixels is d/√2, and thus, signal components having frequencies up to (½√2)fs can be collected according to the sampling theorem.
In FIGS. 3A through 3D, threshold frequency components that can be collected in the vertical, horizontal, and oblique directions are indicated by the voided columns and half-tone columns.
The spatial frequency characteristics of the G, R, and B pixels are shown in FIG. 4. FIG. 4 shows that, when the sampling rate is set to be the pixel pitch d (=1/fs), the spatial frequency characteristics of the G pixels exhibit the resolution up to ½fs in the vertical and horizontal directions and up to (½√2)fs in the oblique 45° direction and the spatial frequency characteristics of the R pixels exhibit the resolution up to ¼fs in the vertical and horizontal directions and up to (½√2)fs in the oblique 45° direction, i.e., signal components up to the above-described threshold frequency can be collected.
Bayer Pixel Shifted Pattern
In addition to the above-described Bayer pattern, the pattern shifted by 45° from the Bayer pattern shown in FIGS. 3A through 3D, such as the pattern shown in FIGS. 6A through 6D, that is, a modified Bayer pattern in which pixels are disposed by being shifted by half the pixel pitch in the vertical and horizontal directions, has been proposed, as disclosed in Japanese Unexamined Patent Application Publication No. 10-262260.
The color pattern generated by shifting the Bayer pattern by 45° is hereinafter referred to as the “Bayer pixel shifted pattern”. In this Bayer pixel shifted pattern, since the sampling rate results in d/√2, which is 1/√2 times as high as the sampling rate d of the Bayer pattern, higher resolution can be obtained compared to that of the Bayer pattern.
From another point of view, if the same resolution is required in the Bayer pattern and in the Bayer pixel shifted pattern, the sampling rate of the Bayer pixel shifted pattern can be increased by √2 as large as that of the Bayer pattern. In other words, by using the Bayer pixel shifted pattern, the same resolution can be obtained with a smaller number of pixels than that in the Bayer pattern. As a result, the pixel aperture can be increased so that the photo-sensitivity of the pixels can be enhanced, thereby obtaining signals having a high signal-to-noise (S/N) ratio.
However, the Bayer pixel shifted pattern can exhibit high resolution only for achromatic subjects. The reason for this is as follows.
FIG. 5 illustrates color coding of the Bayer pixel shifted pattern.
The G pattern from which only the G pixels are extracted from the Bayer pixel shifted pattern is shown FIGS. 6A and 6B. Since the sampling rate for the G pixels in the vertical and horizontal directions is √2d, which is larger than the sampling rate d for the G pixels in the vertical and horizontal directions in the Bayer pattern, the resolution in the Bayer pixel shifted pattern is lower than that in the Bayer pattern. On the other hand, since the sampling rate d for the G pixels in the 45° oblique direction is smaller than the sampling rate d/√2 in the 45° oblique direction in the Bayer pattern, the resolution is higher than that in the Bayer pattern.
Similarly, the resolution of the R pixels and the B pixels is considered. Since the pixel pitches for the R pixels and the B pixels are the same, only the resolution of the R pixels is described below.
The R pattern from which only the R pixels are extracted from the Bayer pixel shifted pattern is shown in FIGS. 6C and 6D. The sampling rate for the R pixels in the vertical and horizontal directions is √2d, and the sampling rate for the R pixels in the oblique direction is 2d.
In FIGS. 6A through 6D, threshold frequency components that can be collected in the vertical, horizontal, and oblique directions are indicated by the voided columns and half-tone columns.
The spatial frequency characteristics of the G, R, and B pixels are shown in FIG. 7. Upon comparing FIG. 7 with FIG. 4, it is seen that the spatial frequency characteristics of the Bayer pixel shifted pattern are the same as those shifted from the spatial frequency characteristics of the Bayer pattern by 45°.
To enhance the effective integrity of pixels including photoelectric transducers, some solid-state imaging devices use the following so-called “oblique pixel pattern”. In this oblique pixel pattern, from a matrix pixel pattern, even-numbered column pixels are displaced from odd-numbered column pixels in the column direction by about ½ the pixel pitch and even-numbered row pixels are displaced from odd-numbered row pixels in the row direction by about ½ the pixel pitch. When a color filter is disposed on a solid-state imaging device having this oblique pixel pattern, the color coding of the Bayer pattern is shifted by 45°, as shown in FIG. 8.
In a CMOS image sensor having an oblique pixel pattern, when line-sequentially reading pixel signals, in a pixel region 101 in which pixels 100 are obliquely disposed, as shown in FIG. 9, a horizontal pixel drive line group 105, each drive line being connected to the pixels 100 in the two zigzag lines, is driven by a vertical selection circuit 106, and signals of the pixels 100 of the selected zigzag lines via the horizontal pixel drive line group 105 are stored in column processing circuits 103, each being disposed for one column, via a vertical signal line group 102, each vertical signal line being disposed for one pixel column. The signals of the pixels 100 stored in the column processing circuits 103 are then sequentially read out to a horizontal signal line 108 via a horizontal selection switch group 107, the switches being sequentially selected by a horizontal selection circuit 104.
In this reading method, the reading speed is fast since many pixel signals can be read out by one reading operation, but on the other hand, it is necessary that the pixel signals in two adjacent rows be read out at the same time, which is less flexible. Accordingly, when performing the pixel addition in the color coding of the oblique pixel pattern shown in FIG. 8 generated by shifting the Bayer pattern by 45°, unlike the pixel addition in the color coding in the Bayer pattern, the resulting color pattern of the added signals becomes different from the original color pattern while finding it difficult to maintain the same color spatial repeat pattern and the same pitch ratio in the vertical, horizontal, and oblique directions.
In another reading method in a CMOS image sensor having an oblique pixel pattern, as shown in FIG. 10, in a pixel region 201 in which pixels 200 are obliquely disposed, a horizontal pixel drive line group 205, each pixel drive line being disposed for one pixel row, is driven by a vertical selection circuit 206, and signals of the pixels 200 of the selected rows via the horizontal pixel drive line group 205 are stored in column processing circuits 203, each being disposed for two zigzag columns, via the vertical signal line group 202, each signal line being connected to the pixels 200 in the same two zigzag columns. The signals of the pixels 200 stored in the column processing circuits 203 are then sequentially read out to a horizontal signal line 208 via a horizontal switch group 207, the switches being sequentially selected by a horizontal selection circuit 204.
In this reading method, it is difficult to implement the fast reading operation since pixel signals can be read out only line by line. Additionally, the pixel signals in the adjacent odd-numbered row and even-numbered row are read out via the same vertical signal line in the vertical signal line group 202 and are processed in the same column circuit 203. Thus, when performing the pixel addition in the color coding of the pixel pattern shown in FIG. 8 shifted from the Bayer pattern by 45°, the resulting color pattern of the added signals becomes different from the original color pattern while finding it difficult to maintain the same color spatial repeat pattern and the same pitch ratio in the vertical, horizontal, and oblique directions.
Differences between the Bayer pixel shifted pattern and the Bayer pattern, which is a typical example of known color patterns, are described below.
In the Bayer pixel shifted pattern, since the sampling rate is 1/√2 times as large as that of the Bayer pattern, pixel information twice as much as that of the Bayer pattern can be obtained as long as the Bayer pixel shifted pattern is used for achromatic subjects. That is, higher resolution can be obtained in the Bayer pixel shifted pattern. In other words, the use of the Bayer pixel shifted pattern exhibits the same resolution as that of the Bayer pixel with a smaller number of pixels, which makes it possible to increase the aperture of the pixels, thereby increasing the pixel photo-sensitivity, i.e., the S/N ratio.
In terms of only the G pixels, which are primary components for generating luminance (Y) components, the sampling rate of the Bayer pixel shifted pattern in the vertical and horizontal directions is larger than that of the Bayer pattern. This means that the resolution of the G pixels in the Bayer pattern in the vertical and horizontal directions is higher than that of the Bayer pixel shifted pattern. In other words, as far as the resolution of the G pixels in the vertical and horizontal directions is concerned, the Bayer pixel shifted pattern is inferior to the Bayer pattern.
To overcome this point, when imaging an achromatic subject, the RGB balance is adjusted in a camera signal processing system, i.e., a gain is applied so that the RGB levels become the same. Then, luminance (Y) components are generated, assuming that the R and B levels are equal to the G level, and the sampling rate of the Y components is handled as (1/√2)d, thereby implementing higher resolution than the Bayer pattern in all the vertical, horizontal, and 45° oblique directions.
However, the above-described processing is effective only for achromatic subjects, and if the same processing is performed on chromatic subjects, it is difficult to obtain high resolution. Additionally, when the level balance is deviated, if the processing is performed assuming that the R and B levels are equal to the G level, it is difficult to perform correct interpolation processing in the camera signal processing system, resulting in the occurrence of false colors.
In view of the above-described background, it is desirable to provide a drive method for a solid-state imaging device, a solid-state imaging device, and an imaging apparatus in which, after adding pixels in an oblique pixel pattern, the original color pattern can be maintained without changing the color spatial repeat pattern or the pitch ratio in the vertical, horizontal, and oblique directions.
It is also desirable to provide a solid-state imaging apparatus and an imaging apparatus that achieve high resolution both for achromatic subjects and chromatic subjects without causing false colors.